"- There are spaces with which Lula should not look to compete - IE policy enforcement engines - But there is a lot of alignment opportunity - how?" If there is a lot of alignment with policy enforcement, should that be an add-on or another function of the tool? By that I mean, if we are doing 60-70% policy enforcement how much more does it take for it to be 100%. The numbers are not correct just for representation, still going through the tool and learning it.

Great question @Alvin-zzz - Current thoughts are certainly to integrate - but not replicate enforcement. IE if we can enforce compliance validation through a policy enforcement engine - we should attempt to do so. But we do not want to mimic or replicate a policy enforcement engine as the market has mature offerings already available.

"- There are spaces with which Lula should not look to compete - IE policy enforcement engines - But there is a lot of alignment opportunity - how?" If there is a lot of alignment with policy enforcement, should that be an add-on or another function of the tool? By that I mean, if we are doing 60-70% policy enforcement how much more does it take for it to be 100%. The numbers are not correct just for representation, still going through the tool and learning it.

Great question @Alvin-zzz - Current thoughts are certainly to integrate - but not replicate enforcement. IE if we can enforce compliance validation through a policy enforcement engine - we should attempt to do so. But we do not want to mimic or replicate a policy enforcement engine as the market has mature offerings already available.

I understand, I'll research an open source policy enforcement engine that would be a good fit or a similar tool that we can integrate or add to Lula.

"- There are spaces with which Lula should not look to compete - IE policy enforcement engines - But there is a lot of alignment opportunity - how?" If there is a lot of alignment with policy enforcement, should that be an add-on or another function of the tool? By that I mean, if we are doing 60-70% policy enforcement how much more does it take for it to be 100%. The numbers are not correct just for representation, still going through the tool and learning it.

Great question @Alvin-zzz - Current thoughts are certainly to integrate - but not replicate enforcement. IE if we can enforce compliance validation through a policy enforcement engine - we should attempt to do so. But we do not want to mimic or replicate a policy enforcement engine as the market has mature offerings already available.

I understand, I'll research an open source policy enforcement engine that would be a good fit or a similar tool that we can integrate or add to Lula.

In order to provide more insight - the current execution of Lula leverages Kyverno as a library for performing validation against Kubernetes configuration. Kyverno is one of the primary Kubernetes Policy Enforcement Engines (it's being targeted as the default engine for Big Bang 2.0).

so is the overall goal to generate OSCAL SDK for golang so you can embed it into golang apps?

does gocomply oscalkit (ie generated from their metaschema golang) not satisfy that need? or is the goal to restart since gocomply oscalkit is 2 years old and seemingly not maintained, ie is the start-from-clean-slate the most important goal (since presumably lots of design forces have changed in the past years)? or is there interest in just refreshing gocomply's golang and maintaining it?

The relationship between components and platforms (and how an end-user would leverage component models for an SSP)

Isn't this defined in OSCAL itself? ie components are defined in CDefs and provide system functionality, and in the SSP model included as providing "implemented-by" security capabilities?

the definition of "platforms" may not be well defined - other than grouping of components. this reminds me of a NIST repo thread about whether SSHD is a component or a system. seems like similar (nuanced and often disagreed on by equally knowledgeable practitioners) discussion. (I could argue contrasting approaches of equal merit.)

The value that a single application (optimally open source) could benefit by collaborating/producing OSCAL

ignoring the above (ie what in fact is an "application", is it a system, a component, etc) ... the benefits of collaboration by a team of devs creating or generating from some metadata an OSCAL artifact would be

1. greater understanding of their own security and compliance wrt. a very comprehensive catalog of controls, which arguably is 99% of the value, likely to stimulate a lot of good discussion and possibly some proactive bug fixes and security enhancements; the mere fact it is OSCAL output vs. chickn scratch on a napkin is irrelevant (but handly for...)

2. having that OSCAL artifact opens the project up to greater reuse by OSCAL-aware teams, GRC tools, CSPs, and agencies who can reuse that artifact in part or wholesale. Given the choice of 2 projects, let's say for example OPA vs Kyverno, if one has OSCAL artifacts and the other doesn't, I might be much more inclined to find a reason to use the one WITH vs the one without. Not the only reason to adopt or reject some open source tool, but a thumb on the scale for some.

The value to end-users for inheritable controls with continuous-compliance validation

depends on which end-users you mean - auditors may love it (or hate it if it means they have to change how they do their job using code instead of word docs and they don't like coding); devops will probably be indifferent insofar as they will see the value of easy to understand yaml based and SDK supported controls, but that is probably offset by having less room for discussion and interpretation and more check-the-box work for them to do and more security related tooling to learn and manage instead of adding new features; project managers will likely like it because they can check off some high level "compliance" checkbox quickly.

A security architect could start building more logic around how controls are implemented and be able to overlay threat intel and other annotations to measure the effectiveness of controls. in short - "shifting" controls left opens up a whole new approach of defensive capability engineering in code that can be reused and recycled across different applications and systems.

Vendors may not like this since it makes it harder to justify expensive single solution tools, and easier to replace one component with another if the capabilities are standardized and commoditized.

Overlap between compliance and policy is quite heavy - leveraging policy to perform compliance-validation potentially improves both

I'd argue that these are actually not actually very overlapping - once you arbitrarily redefine terms :) Elsewhere others have referred to this as big-P "Policy" and little-p policy - Policy is a human readable artifact used for compliance activities; policy is a machine readable artifact used for configuration managment.

in a similar vein, big-C Compliance is the 1) mapping of regulation to specific documentation 2) the mapping and implementation of some process (automated or manual) of doing what you say in that documentation, and 3) the CRUD of documentation artifacts, inclusive of evidence and reviews, etc.

Little-c compliance is simply diffing actual vs. expected of a particular technical knob or lever set.

therefore whether my K8s Volumes are encrypted or not in my definition is not only a "C-ompliance" issue. but is a "c-ompliance" issue, too.

The C-ompliance issues are: a) did I define somewhere that I am using K8s volumes and they need to be encrypted based on some risk model b) did I track that in some Jira card to show that the application or infra was reviewed against that requirement and not allowed to be deployed in IaC until that was true, and c) did I examine or test that the encryption works as advertised (or inherit some documentation from say a FIPS 140 cert that proves it should). None of these overlap with what I would call K8s configuration and configuration "p"olicy management.

The c-ompliance issues are did I have my admission controller check for volume encryption? do I see runtime resources with those settings? can I query and report on those? Can anyone manually touch prod to drift things?

so in short I would restate your generally true proposition slightly more precisely:

Linking C-ompliance controls to P-olicy definitions and then forward engineering specific and fully enumerated configuration requirements corresponding to each P-olicy requirement so that one can create a complete bundle of logic check snippets aka p-olicy which exhaustively cover (as in code coverage) all specified system configuration values at 1) definition time, 2) deployment time 3) ongoing runtime 4) report or diff time....to perform compliance-validation-as-configuration-diff-checks which then can map back to the defined P-olicy and thus the C-ompliance controls.

yes, as Martha Stewart might say, this would be a Good Thing!

policy engine to perform configuration queries provides an interfacing language (rego etc) to utilize in OSCAL validation rules

Agree, p-olicy engine should perform config queries

Also agree (and showed a stupid simple artificial example in the mini-workshop) using Rego to do OSCAL rules could be useful. that said, precisely how you use Rego (or any other language) is where the devil's in the details. if it just doing a lookup then arguably it's not useful and less maintainable and harder to teach others than just hardcoding specific test definition IDs in a JSON map and maintaining the json map in github. Or using CQL. There has to be a specific reason to use Rego vs simpler static mapping expressions. I myself might use simple json maps in some cases, Rego in some cases, and a solver or proof system in others.

Should this be application source or orchestration

I predict it will evolve to be both simultaneously. but initially in the source repo makes more sense as we crawl, before walking...

build out some examples that provide exposing compliance information via REST/gRPC

someone was doing REST...forget who (something "dynamics" maybe - might have been a mini-workshop). but yes lots and lots of examples. The Policy WG will be contributing OSCAL examples soon - maybe a REST if folks want one.

Kubernetes Native resources may provide one solution for deployment of the compliance data

The Policy WG clearly thought that having PolicyReports was useful. And of course Gatekeeper has its own resources. I myself am not so sure in the long term vs. streaming out data into some better system, which might explain the bifurcation of adoption of the Report CR. I think for small data sets, the CR makes sense. For larger domains, with richer, complex data, not so much. Probably so many use cases that both are useful to someone.

perform validation of the Infrastructure that it has connectivity-to

I presume you have seen CloudCustodian?

IaC will use a component-definition OSCAL much like Kubernetes

I think having OSCAL <-> AWS CDK for example would be great! presumably this can already be done via cdk <-> golang/python/nodejs <-> OSCAL in some janky hack or another?

Hey @sunstonesecure-robert ! Appreciate the time you spent running through these notes and providing feedback. Much of the current understanding still needs maturation from expertise such as yourself to help guide development towards valid execution.

Many notes to digest here - I'll start with the first and make my way through questions and the statements provided.

does gocomply oscalkit (ie generated from their metaschema golang) not satisfy that need? or is the goal to restart since gocomply oscalkit is 2 years old and seemingly not maintained, ie is the start-from-clean-slate the most important goal (since presumably lots of design forces have changed in the past years)? or is there interest in just refreshing gocomply's golang and maintaining it?

oscalkit would have some definitive overlap at certain layers of the code. The current problem is - as noted - the GoComply devs have indicated that both that SDK and related tooling are unsupported and out-of-date. The modifications required would require more investigation than we have done for Metaschema to date - and the related material for Metaschema language support still requires some maturation as well.

We'd like to assist NIST with that endeavor while building out more logic towards Kubernetes and other domain validation. Current thought is to mimic methods to establishing datatypes similar to Trestle to provide a baseline - then commit development cycles to Metaschema and the experimentation with contributing to GoComply if possible.

I believe the other half to this question lies in the difference between oscalkit and Lula - primarily being that we would like to see this as a continuous compliance engine - starting with Kubernetes. A controller/operator runtime that is deployed to the environment to perform validation and report generation to GRC tooling. Positioned uniquely from the perspective of portable units of compliance information that can be aggregated and validated for GRC tooling visualization.

There would be ties to Lula as a CLI - I believe there is an inherent value statement to the ability to perform validation prior to introduction to a regulated environment - so if the components delivered to an environment were immutable - then there would be workflows worth pursuing.

as always - would love to bounce ideas back and forth!

@sunstonesecure-robert After digesting your comment more, I think the information you provided helped in my understandings of some concepts that I have heard being discussed in the Policy-WG meetings. Big-P little-p policy and Big C little c Compliance are some perspectives I'm going to chew on further.

Largely I don't have any points of disagreement for what you are saying (it actually helped me learn and solidify some other concepts along the way). As we set-out to define Lula runtime and integration of components - i'd love to get your perspectives on the choices and stances that the tool takes on issues surrounding Policy/Compliance (in all forms). I imagine your experience could help reduce potential pit-falls as we make major architectural decisions - which will be introduce through ADR's for review.

Just to answer the remaining question:

I presume you have seen CloudCustodian?

Yes! it's on the radar as (at a minimum) a learning objective for looking to see how this might be replicated for detecting when something has changed or drifted.

I think having OSCAL <-> AWS CDK for example would be great! presumably this can already be done via cdk <-> golang/python/nodejs <-> OSCAL in some janky hack or another?

Hopefully not too-janky 😉 - but I don't disagree. I need to write a summary around the stance that Lula takes in performing validation for compliance purposes. As long as Lula can be built to allow the user to develop complex (when required) validation - then Lula should care less about the validity of those statements and focus more on the accuracy of validating said configurations. Establishing provenance for a given ruleset that validation is to be performed against is another area that needs to be discussed.

This needs to be put more succinctly - so I will work on that. I'm going to merge this in - but would love to chat more. Feel free to drop changes in PR's and/or just filing issues to establish traceable conversations toward refining the vision.